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Abstract 

Measuring the transmitted information in metric-based clustering 
has become something of a standard test for the performance of a 
spike train metric. In this comment, the recently proposed L v Victor- 
Purpura metric is used to cluster spiking responses to zebra finch 
songs, recorded from field L of anesthetized zebra finch. It is found 
that for these data the C p metrics with p > 1 modestly outperform 
the standard, p = 1, Victor-Purpura metric. It is argued that this is 
because for larger values of p, the metric comes closer to performing 
windowed coincidence detection. 



1 Introduction 

There are a number of reasons why metrics are believed to be useful for 
studying spike trains. It can be argued that it is the most general, useful, 



mathematical framework for spike trains Victor and Purpura 1997 , it may 



be possible to find a manifold of spike trains using local linear methods and, 
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of most relevance to the present discussion, it is possible that studying spike 
train metrics is a useful approach to understanding how content is coded in 
spike trains. 

The Victor-Purpura metric is an edit distance metric on the space of 
spike trains. The distance between two spike trains is, in effect, calculated 
as the cost of changing one spike train into the other by adding, deleting, or 
moving spikes, with an individual cost for each type of elementary moves. In 
particular, a cost of one is associated with adding or deleting a spike and a 
cost of qSt with moving a spike a time St: the distance is 

d(u, v; q) = min 7 c 7;g (u, v) (1) 

where c 7 (u, v) is the cost of a sequence of elementary moves 7 and is calcu- 
lated by adding the cost of all the elementary moves in the sequence. The 
minimum is taken over all sequences 7 changing u to v. The parameter 2/q 
gives an important timescale for the metric: it is never worthwhile to move 
a spike more than 2/q since it would be cheaper to delete the spike from one 
temporal location and to add it to the other. This gives a timescale that, 
roughly speaking, separates jitter from unreliability in comparing spikes in 
the spike trains. However, the Victor-Purpura metric does more than this; 
it explicitly pairs up those spikes that can be thought of as being related by 
jitter. 

The Victor-Purpura metric has an I 1 character: the cost simple 
linear sum of the individual costs of the individual moves. The general- 
ization proposed in |Dubbs et al. 2009] changes this to an Z p -like sum. For 
notational convenience, from now on the set of sequences will be restricted: 
the minimum sequence will never involve moving a spike after it has been 
added, it would always be cheaper to add the spike in the correct location, 
similarly, spikes are never moved before they are deleted. It is also specified 
that spikes can only be moved once. With these restrictions, the sequence 
7 can be considered to be an unordered set made up of deletions, additions 
and moves. Let 

7 = qU/i (2) 

where a is the set of additions and deletions and fi is the set of moves. The 
elements of \i are pairs of spikes (u, v), with u G u and v G v, that are related 
by jitter. 
Now, 

c 7;(J (u, v) = \a\ + q\u — v\ (3) 
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This is generalized in [Dubbs et al. 2009] to 

c w (u,v) = l|a|+ Yl 1 P \ U - V \ P \ ( 4 ) 

\ (u,v)efi J 

with p > 1, to give the C p Victor-Purpura metric 

d(u, v; q, p) = min 7 c 7; q iP (u, v) (5) 

where the minimum is taken over sequences 7 changing u to v and satisfy- 
ing the restrictions specified above. An algorithm for calculating this quan- 
tity is given in |Dubbs et al. 20 09J; the existing algorithms used to calculate 
the Victor-Purpura metric, [Victor et al. 2009] . for example, could also be 
adapted to p > 1. 



2 Evaluating the metric using clustering 

In the data that will be considered here, spike trains were recorded from field 
L of the auditory fore-brain of anesthetized zebra finch during playback of 
20 con-specific songs with each song repeated ten times, to give a total of 
200 spike trains. These spike trains, and the experimental conditions used 
to produce them, are described in |Narayan et al. 2006 Wang et al. 2007 



and they have previously been used to compare metrics in |Houghton 2009 



Houghton and Victor 2009] . The key point, in this context, is that a good 



metric, one which depends on the content encoded in the spike train, should 
measure a smaller distance between spiking responses to the same song than 
between responses to different songs. One motivation for looking at spike 
train metrics is that scoring metrics in this way, and examining how good 
metrics measure distance, might reveal details about how content is encoded. 

The usual method for evaluating how well a metric succeeds in this way is 
to calculate the transmitted information Victor and Purpura 1996 1. There 



are two ways to cluster the spike trains: a true clustering, based on the 
identity of the song that elicited the response and an estimated clustering, 
based on the metric distances between the responses. Roughly speaking, the 
transmitted information quantifies the amount of information about the true 
clustering given by the estimated clustering. 

Calculating the transmitted information relies on the calculation of a 



confusion matrix N. This is described in detail in Victor and Purpura 1996 , 
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described specifically for the data considered here, N is a 20 x 20 matrix of 
integers where the entry iVy counts how many of the spike trains from the 
ith true cluster is closest, on average, to the spike trains in the jth true 



cluster. As in Victor and Purpura 1996 this averaging is carried out as a 



root-mean-square averaging to under-weigh outliers. If the metric clustering 
is close to the true clustering, N will be nearly diagonal. The transmitted 
information gives a measure of this, it is 



nln 20 



% ( ln N tj - In N kj ~ In Nik + ln n ) 

ij \ k k / 



(6) 



where n = ^ . iV^-. All sums are from one to 20: the number of clusters. 

The factor of ln20 normalizes h and so that it takes values between zero 
and one; a low value corresponds to poor clustering and a value near one 
corresponds to a near-diagonal confusion matrix. 

The main purpose of this comment is to evaluate the C p Victor-Purpura 
metric by calculating h for the zebra finch data previous used to evaluate 
other spike train metrics |Houghton 2009 , Houghton and Victor 2009] . The 



data set contains 24 sets of spike trains, corresponding to 24 recording sites 
from multiple birds. Although the songs themselves are of different lengths, 
all of the songs are at least a second long and, in each case, the spike train 
is truncated to one second, starting at song onset. 

The performance of the £,„ Victor-Purpura metric is illustrated in Fig. [TJ 
A remarkable feature is that h rarely decreases as p is increased. Other multi- 
parameter metrics show a mixed effect; although changing a parameter might 
increase the average h across the sites, there will typically be many individual 
sites where h decreases. In the case of p, while the average improvement as p 
increases is modest and a small number of sites show no change; there is no 
site where the clustering is negatively affected when p is changed from one 
to two, three do show a tiny decrease between p — 1 and p = 1.5. h increases 
for 19 sites between p — 1 and p = 2 and for 15 between p = 2 and p — 3, 
the improvement gets less and less and appears to have plateaued by p = 10. 
The average performance is plotted against p in Fig. p. The optimal value 
of q does not vary significantly as p changes; however, the range of q that 
produces the optimal performance does get wider. 
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Figure 1: Performance as p changes. A, B and C correspond to p — 1, p = 2 
and p = 10. Each horizontal line runs from zero to one. For clarity, there are 
tiny gaps at 0.25, 0.5 and 0.75. Each small vertical dash marks the h value for 
one site where for each site the optimal value of q is used. The large vertical 
dash marks the average, 0.732, 0.751 and 0.761 respectively. To indicate the 
range over which h values vary, two other metrics are plotted for comparison. 
D corresponds to the van Rossum metric [van Rossum 20 01J. with each site 
using its optimal value of r. E corresponds to the synapse metric, with each 
site using its optimal value of r and \i |Houghton 2009| . Finally, F corre- 
sponds to the L 1 van Rossum metric, with a boxcar filter; this metric mea- 
sures a very similar distance to the p = 1 metric Houghton and Sen 2008 . 
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Figure 2: The average performance plotted against p. h has been calculated 
for a range of p values using the optimal value of q for each site and the 
average value of h is plotted against p. 

3 Discussion 

It is interesting to compare this metric to another generalization of the Victor- 
Purpura metric Victor and Purpura 1996 , Victor and Purpura 1997] . The 



linear distance function q5t can be replaced by any convex function, f(\5t\), 
of distance: 

c 7;/(-)( u > v ) = l«l + Yl f(\ u ~ v \)- ( 7 ) 

The distance function f{\St\) must be convex to ensure that the triangular 
inequality is satisfied. Now, the function q p 5t p in c 7;giP above is concave for 
p > 1. However, because of the 1/p exponent in d(u, v;q,p), this does not 
cause the same difficulty with the triangular inequality: an interesting prop- 
erty of the C p metrics is that they satisfy the triangular inequality despite 
having a concave distance function. 

As p is increased, q p \5t\ p becomes increasingly concave and spikes located 
at similar times contribute less and less to the total cost and this cost is 
predominately made up of the cost of adding and deleting spikes that are 
not related by jitter. This means that the metric performs more and more 
as a windowed coincidence detector as p increases. The fact that it is always 
beneficial to increase p appears to imply that, for these data, coincidence and 
coincidence detection has a role in the encoding and decoding of content in 
spike trains. 
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Figure 3: Comparing the L 1 boxcar van Rossum metric with the p — 1 
and p = 10 C p metrics. Here h values for the p = 1 and p — 10 metrics 
are graphed against the values for the L l van Rossum metric with a boxcar 
filter with optimal values of the q or r parameter has been used for each 
site. The p = 1 and p — 10 values are marked by a horizontal line, the two 
values for a given site are joined by a vertical line. Since increasing p never 
decreases h between p = 1 and p = 10, the topmost of the two horizontal 
lines corresponds to p = 10. The 'x=y' line is also plotted for clarity. 
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The pairing of near coincident spikes is the key distinction between the 
Victor-Purpura metrics and the van Rossum metrics, a family of spike train 
metrics which are calculated by comparing reconstructed rate functions. In 
fact, computationally, an L l van Rossum metric with a boxcar filter measures 
a very similar distance to the Victor-Purpura metric Houghton and Sen 2008 1 
and, as seen in Fig. [31 has a very similar performance. The way in which C p 
Victor-Purpura metric performance increased with p seems to indicate that 
the significant temporal structure of spikes is not fully accounted for by the 
temporal structure of a spike rate. 
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